Skip to content

Conversation

iowillhoit
Copy link
Contributor

@iowillhoit iowillhoit commented Jun 21, 2022

What does this PR do?

When running force:source:manifest:create --fromorg, the singleRecordQuery method to retrieve StandardValueSet was having connection errors, added retry logic.

What issues does this PR fix or reference?

@W-11303606@

Functionality Before

Connection errors would cause some queries to fail. The result was different package.xml file almos every time you ran force:source:manifest:create --fromorg YOUR_ORG

package-xml-diffs-720.mov

Functionality After

Failed queries retry, the package.xml is the same every time

Testing

  • Pull and cd into this source-deploy-retreive branch
  • yarn build
  • yarn link (if needed)
  • Pull and cd into plugin-source
  • yarn link @salesforce/source-deploy-retrieve
  • sfdx plugins:link .
  • Create a new Dreamhouse org
    • git clone git@github.com:trailheadapps/dreamhouse-lwc.git dreamhouse-testing-standard-value-set
    • cd dreamhouse-testing-standard-value-set
    • sfdx force:org:create -s -d 1 -f config/project-scratch-def.json
    • Get the username
  • Run sfdx force:source:manifest:create --fromorg THE_USERNAME
  • Commit the created package.xml
  • Run the command several more times and use git diff to confirm not changes occur over multiple runs
    • Note: you can see the retries by adding the following here
,
{
  logger: (msg) => {
    console.log(msg);
  },
}

Now unlink everything! 🔥🔗

@iowillhoit iowillhoit requested review from a team as code owners June 21, 2022 20:07
@iowillhoit iowillhoit requested a review from angela-young June 21, 2022 20:07
command: |
npm install shx -g
shx rm -rf node_modules/@salesforce/source-deploy-retrieve
working_directory: node_modules/@salesforce/source-tracking
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a common thing to do for circular deps?
Does this not mean that for the tests source-tracking would be using the current version of SDR, but after deploy it'd be using whatever version it specifies in it's package.json?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they're not circular, just transitive. plugin-source depends on SDR and STL and STL depends on SDR.

When we build the CLI, we bump both SDR and STL via yarn resolutions to their latest version.

This tests the changes on this branch everywhere that SDR is used. Imagine a bug in SDR that causes problems for STL, but you don't catch it because the nuts are using an STL with a different (bug-free) SDR. Wouldn't want to miss that!


if (error.lastError && error.lastError.message) {
this.logger.debug(error.lastError.message);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a few unit tests to cover the conditions added here?

  • retries and succeeds
  • Hits the retry limit (seemed a bit like exercising the retry lib, but would be good to have a test to verify what happens when it hits the limit). Could just stub the failure condition early verse having to actually call until the lib fails.
  • Not Retryable error thrown

It would make the unit testing a bit cleaner if you refactored the standardValueSetRecord specific handling out into a method

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some tests @gbockus-sf 👍

shiki "^0.10.1"

typescript@^4.1.3, typescript@^4.4.3:
typescript@^4.1.3, typescript@^4.4.3, typescript@^4.7.4:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like several of the deps are now out of sync with what's in @salesforce/dev-scripts. Just verifying that's the way to go.

@jayree
Copy link
Contributor

jayree commented Jul 1, 2022

@iowillhoit I know these problems from the past. However, I have not seen this issue since api version 50. What error messages appear in the debug log?

throw err;
}

return result;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you return the result from the try {} block, that would avoid the let , the assignment, and possibly the await

@iowillhoit
Copy link
Contributor Author

@iowillhoit I know these problems from the past. However, I have not seen this issue since api version 50. What error messages appear in the debug log?

@jayree Here are the two types of errors I am seeing:
TLS Connection:

request to https://customer-inspiration-1104-dev-ed.my.salesforce.com/services/data/v55.0/tooling/query?q=SELECT%20Id%2C%20MasterLabel%2C%20Metadata%20FROM%20StandardValueSet%20WHERE%20MasterLabel%20%3D%20%27ProcessExceptionStatus%27 failed, reason: Client network socket disconnected before secure TLS connection was established

Connection Reset:

FetchError: request to https://customer-inspiration-1104-dev-ed.my.salesforce.com/services/data/v55.0/tooling/query?q=SELECT%20Id%2C%20MasterLabel%2C%20Metadata%20FROM%20StandardValueSet%20WHERE%20MasterLabel%20%3D%20%27WorkTypeGroupAddInfo%27 failed, reason: read ECONNRESET
    at ClientRequest.<anonymous> (/Users/ewillhoit/dev/source-deploy-retrieve/node_modules/node-fetch/lib/index.js:1491:11)

@WillieRuemmele
Copy link
Member

QA NOTES:


✅ : following instructions in post, generated manifest 5 times, no differences
✅ : sanity check --metadata StandardValueSet 5 times, no differences
✅ : unlink SDR/PS see varying results in package.xml

@WillieRuemmele WillieRuemmele merged commit bb27b8d into main Aug 3, 2022
@WillieRuemmele WillieRuemmele deleted the ew/standard-value-set-connection-issues branch August 3, 2022 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants